Comparing Vision APIs

This notebook features the various computer vision APIs that pliers interfaces with. These include the Google Vision, Clarifai, and Indico APIs. To compare their perfomance, image recognition features are extracted from an image of an apple.



In [1]:

    
from pliers.tests.utils import get_test_data_path
from os.path import join
from pliers.extractors import (ClarifaiAPIImageExtractor, IndicoAPIImageExtractor, GoogleVisionAPILabelExtractor)
from pliers.stimuli.image import ImageStim
from pliers.graph import Graph



In [2]:

    
# Load the stimulus
stim_path = join(get_test_data_path(), 'image', 'apple.jpg')
stim = ImageStim(stim_path)



In [14]:

    
# Configure extractions
clarifai_ext = ClarifaiAPIImageExtractor()
indico_ext = IndicoAPIImageExtractor(models=['image_recognition'])
google_ext = GoogleVisionAPILabelExtractor()









    



WARNING:googleapiclient.discovery_cache:No module named locked_file
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/googleapiclient/discovery_cache/__init__.py", line 41, in autodetect
    from . import file_cache
  File "/Library/Python/2.7/site-packages/googleapiclient/discovery_cache/file_cache.py", line 36, in <module>
    from oauth2client.locked_file import LockedFile
ImportError: No module named locked_file



In [15]:

    
# Run extractions
clarifai_res = clarifai_ext.transform(stim)
indico_res = indico_ext.transform(stim)
google_res = google_ext.transform(stim)



In [11]:

    
clarifai_res.to_df()









    Out[11]:






  
    
      
      onset
      duration
      apple
      fruit
      food
      juicy
      confection
      health
      delicious
      no person
      ...
      diet
      agriculture
      juice
      isolated
      sweet
      freshness
      vitamin
      tasty
      shining
      color
    
  
  
    
      0
      NaN
      NaN
      0.997886
      0.992864
      0.989317
      0.981853
      0.969632
      0.966955
      0.962412
      0.960222
      ...
      0.952635
      0.950934
      0.932082
      0.904409
      0.901952
      0.901597
      0.878773
      0.876409
      0.874465
      0.861764
    
  

1 rows × 22 columns



In [32]:

    
df = indico_res.to_df()
df.loc[:, df.sum() > 0.5]









    Out[32]:






  
    
      
      image_recognition_pomegranate
    
  
  
    
      0
      0.843756



In [13]:

    
google_res.to_df()









    Out[13]:






  
    
      
      onset
      duration
      apple
      food
      fruit
      produce
      plant
      rose family
      land plant
      flowering plant
      rose order
      manzana verde
    
  
  
    
      0
      None
      None
      0.948273
      0.937953
      0.917061
      0.89393
      0.883235
      0.791557
      0.760017
      0.72544
      0.587203
      0.529798

Summary

For the apple image, it is clear that the Google and Clarifai APIs perform best, as both have "apple", "food", and "fruit" in the top features. On the other hand, the only Indico API feature with a probability over 0.5 is "pomegranate". Furthermore, the Google API seems to also be less noisy than the Clarifai API, where several object labels have probabilities over 0.9.



In [ ]: